# Long Sequence Processing

A sentence transformer model trained on the PubMed dataset, supporting multiple embedding dimensions, suitable for biomedical text processing.

Ruri v3 is a Japanese general-purpose text embedding model based on ModernBERT-Ja, supporting sequence processing of up to 8192 tokens and delivering top-tier performance in Japanese text embedding tasks.

Sapnous-6B is an advanced vision-language model that enhances perception and understanding of the world through powerful multimodal capabilities.

FANformer-1B is an autoregressive model enhanced with innovative periodic mechanisms for language modeling, featuring 1.1 billion non-embedding parameters and trained on 1 trillion tokens.

CodeModernBERT-Owl is a model pre-trained from scratch, specifically designed for code retrieval and code understanding tasks, supporting multiple programming languages and improving retrieval accuracy.

Zamba-7B-v1-phase1 is a hybrid architecture combining state space model Mamba with Transformer, using Mamba as the backbone network and sharing one Transformer layer every six modules, trained via next-token prediction.

A large-scale BERT model trained from scratch on Cantonese text, suitable for masked language modeling tasks in Cantonese

Mistral-SUPRA is a linear RNN model initialized based on Mistral-7B, combining the functions of Transformer and recurrent models.

Saul-Instruct-v1-GGUF is the GGUF format version of the Equall/Saul-Instruct-v1 model, suitable for text generation tasks and supports multiple quantization levels.

Mamba is an efficient sequence model compatible with transformers, with 790 million parameters, suitable for causal language modeling tasks.

Mamba is a transformer-compatible sequence modeling model with efficient inference capabilities.

Mamba is an efficient language model based on the State Space Model (SSM) architecture, with 1.4B parameters, supporting text generation tasks

A text ranking model based on Mistral-7B, offering multiple quantized versions for efficient inference.

GGUF quantized version of Mixtral-8x7B-v0.1, supporting multiple bit quantization, suitable for text generation tasks.

This is a German/English bilingual model fine-tuned based on Mistral-7B-Instruct-v0.1, quantized in GGUF format with support for multiple quantization levels from 2 to 8 bits.

Mamba-1B is a 1B-parameter language model based on the Mamba architecture, supporting English text generation tasks.

Dolphin 2.5 Mixtral 8X7B is a large language model developed by Eric Hartford based on the Mixtral architecture, fine-tuned on multiple high-quality datasets, suitable for various natural language processing tasks.

Mixtral-8x7B is a pre-trained generative sparse mixture of experts large language model that outperforms Llama 2 70B on most benchmarks.

JAIS-30B is a 30-billion-parameter bilingual (Arabic and English) large language model based on the GPT-3 architecture, utilizing ALiBi positional embedding technology, achieving state-of-the-art performance in Arabic tasks.

Llava v1.5 13B is a multimodal model developed by Haotian Liu, combining visual and language capabilities to understand and generate content based on images and text.

13 billion parameter Arabic-English bilingual large language model based on Transformer architecture, supporting long sequence processing

CodeLlama 34B Instruct is a 34-billion-parameter code generation model released by Meta, based on the Llama 2 architecture, specifically fine-tuned for programming tasks.

Long-sequence Nyströmformer model trained on WikiText-103 v1 dataset, supports sequence processing up to 4096 tokens

Nystromformer model trained on the WikiText-103 dataset, supporting long sequence processing (2048 tokens)

This model is a fine-tuned speech recognition model based on facebook/wav2vec2-xls-r-2b on the common_voice dataset

A pre-trained unbalanced Transformer model for Chinese understanding and generation, supporting various natural language processing tasks

An efficient Transformer model optimized with the Nyström method for handling long-sequence tasks

Pre-trained on BERT-style masked language modeling tasks, supports multimodal Transformer model processing UTF-8 byte inputs

BigBird is a Transformer model based on sparse attention, capable of processing sequences up to 4096 tokens long, suitable for long document tasks.

Asymmetric pre-trained Transformer model for Chinese comprehension and generation tasks

BioBERT is a BERT-based pretrained language model specifically optimized for biomedical text mining tasks, suitable for question answering systems.

YOSO is an efficient Transformer variant that reduces the self-attention complexity from quadratic to linear through Bernoulli sampling attention mechanism, supporting sequence lengths up to 4096.